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This paper summarizes evaluation data of at-risk 
student outcomes within the California State University (CSU) System 
in terms of student retention and academic performance. Findings 
concerning student retention are presented in the following areas: 
retention by ethnicity and by admission basis; retention by entering 
test performance; and the retention rates across cohorts and 
programs. This is followed by an analysis of student performance in 
the Intensive Learning Experience (ILE) remedial program and in the 
orientation program, Summer Bridge; and of student performance on the 
Entry Level Mathematics (ELM) exam and subsequent enrollment and 
grades in baccalaureate mathematics courses. Among the key findings 
are that: (1) passing ELM greatly increased student chances of 
enrolling in and passing baccalaureate math; (2) retention figures 
for ILE students were comparable to overall CSU system figures; and 
(3) many students who fail the ELM exam and retake it go on to pass 
the exam and eventually succeed in their baccalaureate course. 
(GLR) 
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The California State University 
(CSU) system enrolls nearly 350,000 
students on 20 campuses. Within the 
student population, however, 
Hispanic, African-American, and 
other minority students are under- 
represented. Although Hispanic 
youths constitute 23.0 percent of 
California's 18-20 year-olds, only 8.5 
percent are eligible for admission to 
the CSU system; and African- 
Americans make up 8.9 percent of 
the young adult population, but only 
5.5 percent are eligible. In contrast, 
61.3 percent of 18-20 year-olds are 
Caucasian, but nearly 75 percent 
meet the admissions standards 
(California State University Com- 
mission on Hispanic Under-repre- 
sentation, 1984). There is thus a 
compelling need to focus on increas- 
ing university enrollment of African- 
American, Hispanic, recently- 
arrived Asian, and Native American 
students in California. 

Many of the students who do 
enter the system are under-prepared 
academically, are the first generation 
in their family to attend college, or 
are financially disadvantaged. 
Within the CSU system, minorities 
account for over 80 percent of those 
scoring in the lowest quartile on the 
English Placement Test (EFT), and 
nearly 50 percent of those scoring in 
the lowest quartile of the Entry Level 
Mathematics exam (ELM). In 
addition, nearly 70 percent of 
African- American students and 38.0 
percent of Hispanics enter the CSU 
system through a "special admis- 
sion" provision, i.e., they do not 
meet the regular admissions criteria. 



Each of the system's campuses 
has developed various programs to 
address these twin concerns of 
under-representation and under- 
preparation for minority students; 
however, it wasn't until 1986 that 
the Chancellor's Office of the CSU 
took a systemwide approach 
through two specially-designed 
programs: the Summer Bridge and 
Intensive Learning Experience (ILE) 
programs. Both are designed to 
provide basic skills instruction, 
orientation, and advisement to 
entering CSU students who have a 
high potential for dropout or failure. 
And the underlying objective of 
each is to increase the enrollment 
and retention of under-represented 
minorities in the CSU system. 

Summer Bridge is a three-to-six 
week (largely) residential program 
for incoming first-time freshmen. 
The primary focus is on under- 
represented students; but special 
admission students, first generation 
college students, and those partici- 
pating in other equity programs are 
also targeted. Over the summer, 
participants are given review in 
English and mathematics as well as 
academic advising, counseling, and 
orientation to the university. 

The Intensive Learning Experi- 
ence program offers remediation in 
English and mathematics to stu- 
dents who score in the lowest 
quartile on the English and math- 
ematics exams taken by entering 
freshmen. Student arc given a full 
academic year of writing and/or 
math in small classes. 



Evaluation 

From 1986 to 1991, Far West 
Laboratory for Educational Research 
and Development conducted a five- 
year, longitudinal evaluation of the 
two programs (L.F. Guthrie & G.P. 
Guthrie, 1988; L.F. Guthrie, G.P. 
Guthrie, Long, & Boothroyd, 1988; 
L.F. Guthrie, G.P. Guthrie, & 
Tokunaga, 1991). The evaluation 
assessed outcomes for students in 
two broad areas: retention (contin- 
ued enrollment) and academic 
performance. Three cohorts of 
Summer Bridge and ILE students 
were tracked over a five year period. 
Using both qualitative and quantita- 
tive methods, a sample of nearly 
7,000 Summer Bridge and over 
12,000 ILE students across all 
campuses were studied along 
several dimensions. With compari- 
son groups, the total study sample 
included nearly 40,000 students. 

This brief summarizes the 
findings of the evaluation in terms of 
student retention and academic 
performance. First is a summary of 
findings on retention; next, findings 
are presented from the analysis of 
ILE student performance in math- 
ematics through an examination of 
students' performance on the Entry 
Level Mathematics exam and 
enrollment and grades in baccalau- 
reate mathematics courses. 

Student Retention 

Not only are Native Americans, 
African- Americans, and Hispanics 
under-represented in the CSU 
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system, but fewer students from 
these groups persist to graduation. 
While about 50 percent of regularly 
admitted Caucasian students are 
either still enrolled or have gradu- 
ated after five years, the figure for 
African-Americans is only 38.8 
percent. Among special admits, 32.4 
percent of Caucasians and 23.7 
percent of African-Americans were 
retained (California State University, 
Office of the Chancellor, Division of 
Analytical Studies, 1985). 

Retention rates for Summer 
Bridge and ILE students were 
analyzed from several perspectives 
in order that we might identify 
factors associated with greater 
retention. We looked at various sub- 
populations of students as defined 
by campus, ethnicity, admission 
basis, and student major to see 
which have the greatest rate of 
persistence. 

Some caution must be used in 
interpreting retention rates: students 
in the 1980s do not enroi?. continu- 
ously to graduation, but may work 
for one or more terms and return to 
college later. This is especially true 
of those for whom finances are a 
concern. 



Retention of Summer Bridge Students 

Retention of Summer Bridge 
students was examined along the 
dimensions of admission basis, 
ethnicity, and entering ELM score. In 
the following discussion, we also 
make comparisons to the CSU 
systemwide retention rates. 

Retention by Ethnicity and 
Admission Basis 

Figure 1 shows retention/ 
graduation rates by admission basis 
and ethnicity for the 1985 Summer 
Bridge cohort. After five years, the 
highest persistence rates were 
achieved by Asians (53.8 percent) 
and Filipinos (51.6 percent). African- 
Americans (31.3 percent) and Native 
Americans (35.4 percent) had the 
poorest retention /graduation rates. 
Whether students entered through 
regular or special admission basis, 
this pattern across ethnic groups is 
maintained. 

An important goal of the Sum- 
mer Bridge program is to increase 
the retention and graduation of 
under-represented minority stu- 
dents. To explore the program's 
effectiveness in this area, v/e ex- 



tracted retention data on thSse 
students for each campus and found 
that seven campuses retained or 
graduated over 40 percent of under- 
represented minorities after five 
years. 

Retention by Entering Test 
Performance 

A final way in which we have 
examined retention among Summer 
Bridge and ILE students is in 
relation to scores on the Entry Level 
Mathematics exam and the English 
Placement Test. Retention of ILE 
students was examined in relation to 
initial ELM and EPT scores. Because 
of the program's particular emphasis 
on academic performance, these 
analyses are especially important. 

In our earlier report (L.F. 
Guthrie & G.P. Guthrie, 1988), we 
found a positive correlation between 
entering ELM score and retention. 
Table 1 shows five-year retention/ 
graduation rates for the 1985 ILE 
and comparison groups by ELM 
score bands. The first three score 
categories represent the lowest 
quartile for ELM, the ILE-eligibility 
criterion. 1 he score bands show 
progressively higher retention rates. 

We also looked at retention/ 
graduation rates for the group of 
students who were eligible for ILE, 
but not enrolled. For student. : 
having ELM scores, rates were 
generally higher for ILE students 
than for the ILE-Eligible 1 . Across 
score bands, ILE students in the 16- 
21, 33-37, and 38+ score bands had 
higher retention/graduation rates; 
only the lowest band favored the 
ILE-Eligibie group. This suggests 
that ILE math may have a greater 
impact on students' retention than 



1 Note that ELM scores were re- 
ported for only a sub-sample of ILE, 
ILE-Eligible, and ILE-Marginal 
students. About 60 percent of ILE 
and 80 percent of ILE-Eligible and 
ILE-Marginal students had ELM 
scores. 



other remedial math courses. 
Contrary to expectations, ILE 
students' retention rates also ex- 
ceeded those of the group of stu- 
dents who scored marginally above 
the ILE cut-off in all but the 26-32 
score band. 

Now consider comparable 
analyses on initial English Placement 
Test scores and retention/ gradua- 
tion for the 1985 ILE cohoit and 
comparison groups (Table 2). A 
different pattern emerges. Reten- 
tion/graduation rates for students in 
the lowest score bands (120-126; 127- 
135) are greater than for those 
students who scored highest (151+). 
Notice especially that among special 
admission students, only 31.0 
percent of those with passing scores 
were still enrolled, compared to 35.3 
percent in the lowest band. 

Retention Rates Across Cohorts and 
Programs 

Figure 2 shows five-year reten- 
tion/ graduation rates by admission 
basis for Summer Bridge, ILE, and 
the 1983 cohort of first-time fresh- 
men in the CSU system. These 
comparisons reveal the effectiveness 
of both programs in retaining special 
admit students who are from largely 
under-represented minority back- 
grounds. Systemwide, 33.2 percent 
of special admits were still enrolled, 
compared to 33.0 percent for Sum- 
mer Bridge and 3" .8 percent for ILE. 

Student Performance 

A common goal of both Summer 
Bridge and ILE is to improve stu- 
dents' academic performance in the 
university, and more specifically, in 
baccalaureate English and math- 
ematics. The expectations for each 
program are not the same, however, 
given the differences in focus and 
scope. Summer Bridge is designed to 
improve students' orientation and 
adjustment to university life, as well 
as their academic performance. The 
program provides, at most, four-to- 
five weeks of review and instruction. 
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For ILE, on the other hand, the 
primary objective is to ensure 
students' success in math and 
English, and ILE courses may span 
two full semesters. 

In the evaluation of these 
programs, we approached the issue 
of performance in saveral ways. We 
examined students' overall GPA and 
then analyzed their grades in 
baccalaureate English and math. We 
also explored patterns of course- 
taking in English and math by 
measuring students' enrollment in 
remedial, baccalaureate, and other 
courses. This section focuses on the 
relationship between Intensive 
Learning Experience students' 
entering Entry Level Mathematics 



score and success in baccalaureate 
mathematics courses. 

Grade reports covering the first 
four years of enrollment for the 
1985-86 cohort of ILE students were 
examined to determine the propor- 
tion of students who a) enrolled in 
baccalaureate math by the end of 
their fourth year and b) received a 
passing grade. The sample of ILE 
students were first divided into 
three groups based upon their initial 
ELM score: ILE (25 or less), Marginal 
(26-37), and Passing (38 or above). 
Students having no ELM score were 
excluded from the analysis. Figure 3 
shows the number within each 
group who re-enrolled in the 
university, took a baccalaureate 



Table 1 
1985-86 Cohort 

Five-Year Retention and Graduation* by ILE Comparison Groups, 
Admission Basis, and ELM Score Categories 
ILE 

ELM Score Categories 



Admission 
Basis 


N % 


N % 


N % 


N % 


N % 


N % 


N % 


Alternate 


0 0.0 


1 33.3 


2 50.0 


1 25.0 


0 0.0 


7 58.3 


11 37.9 


Regular 


15 36.6 


82 46.3 


107 48.4 


84 45.9 


73 53.3 


291 55.4 


652 50.7 


Special 


32 22.0 


97 31.8 


66 30.7 


62 29.9 


44 42.4 


91 48.4 


392 33.7 


Total 


47 24.7 


180 37.1 


175 39.7 


14/ 37.2 


117 48.1 


389 53.7 


1055 42.5 



Table 2 
1985-86 Cohort 

Five-Year Retention and Graduation* by ILE Comparison Groups, 
Admission Basis, and EPT Score Categories 
ILE 

EPT Score Categories 



Admission 
Basis 


N % 


N % 


N % 


N % 


N % 


N % 


N % 


Alternate 


3 33.3 


5 45.5 


2 20.0 


0 0.0 


1 100.0 


0 0.0 


11 29.7 


Regular 


54 47.8 


187 52.6 


330 61.1 


56 35.9 


85 48.8 


52 39.4 


764 51.9 


Special 


54 35.3 


149 33.6 


125 36.6 


32 29.1 


24 24.7 


14 33.3 


398 33.5 


Total 


111 40.3 


341 42.0 


457 51.3 


88 32.6 


110 40.4 


66 37.5 


1173 43.5 



* Percentages represent the number of students retained or graduated. 
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math course, and received a passing 
grade. 

The analysis reveals a clear 
connection between students' initial 
ELM score and subsequent reten- 
tion, enrollment in baccalaureate 
math, and course success. On each 
measure, the ILE students were less 
successful than either the Marginal 
or Passing groups. A smaller per- 
centage of ILE students re-enrolled, 
fewer took baccalaureate math, and 
very few received a passing grade in 
their math course. 

In interpreting these findings, 
bear in mind that the opportunities 
to enroll in a baccalaureate math 
course were not equal across the 
three groups. While all students in 
the sample had at least one academic 
year in which to take the math 
course, those who passed ELM on 
the first attempt had four full years. 
Students scoring in the bottom 
quartile on ELM, on the other hand, 
may have spent up to two semesters 
in ILE or other remedial courses; 
Marginal students might have 
enrolled in other remedial math 
courses prior to their baccalaureate 
course. Furthermore, a passing ELM 
grade is technically a prerequisite to 



enrollment in baccalaureate math; 
thus, we would expect the course- 
taking rates to be higher among 
those who eventually managed to 
score above 38 on the ELM exam. 

Figures 4, 5, and 6 allow for a 
closer examination of the ILE and 
other students' math experiences in 



the first four years. Figure 4 shows 
that of the 941 ILE students, 431 
(45.8 percent) took ELM only once; 
of those, only 22 (5.1 percent) 
eventually passed baccalaureate 
math. A passing score on the ELM 
exam is a prerequisite to graduation, 
and students can take the exam as 
many times as necessary to pass. 
Thus, many of the students who 
intially failed the exam made further 
attempts. A total of 155 (165 per- 
cent) passed ELM, and 58 of those 
(37.2 percent) went on to pass the 
course. Over 350 students did not 
manage to pass the exam after four 
years, however. From this unsuc- 
cessful group, only 29 (8.2 percent) 
were able to take and pass the 
course. 

This analysis demonstrates the 
importance of passing ELM; even if 
students fail on the first attempt, 
nearly 40 percent of the 155 who 
take the exam again and pass it, 
eventually succeed in their 
baccalaureate course. 

In the Marginal group (Figure 
5), about half the students (272) took 
ELM only once. Compared to the 
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Re-enrolled Took Math Passed Math 



ELM Score 


Total # 


# 


% 


# 


% 


# 
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25 or less 


941 


425 


45.2 


204 


21.7 


109 


11.6 


26-37 


576 


293 


50.9 


244 


42.4 


170 


29.5 


38+- 


604 


386 


63.9 


315 


52.2 


294 


48.7 
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ELM Attempts 


Total # 


# 


% 


# 


% 


# 


% 


1 ELM Only 


431 


153 


35.5 


49 


11.4 


22 


5.1 


1+ ELM (Fail) 


355 


175 


62.5 


59 


16.6 


29 


8.2 


1+ ELM (Pass) 


155 


97 


49.3 


96 


61.9 


58 


37.4 



ILE group, these students per- 
formed much better. Nearly one- 
fourth (24.6 percent) passed the 
baccalaureate math course (67), 
compared to only 5.0 percent of the 
bottom quartile (ILE) students. 
However, for the Marginal group 
students who attempted the exam 
more than once, course enrollment 
and passing rates are similar to 
those of bottom-quartile students 
who took the exam again. Of these 
students, nearly two-thirds eventu- 
ally passed (198), and as with the 
ILE group, about 40 percent of the 
test-passers went on to pass the 
course. 

Figure 6 displays comparable 
data for the highest group — those 
who passed ELM on the first 
attempt. Math course enrollment 
rates for students in this group are 
greater than those for the other two. 
Over half (315) enrolled in baccalau- 
reate math, and nearly as many 
(48.7 percent) were able to pass the 
course. 

This analysis illustrates the 
effect of taking ELM more than once 
on baccalaureate math performance. 



For example, only about 11 percent 
of ILE students who took ELM only 
once managed to enroll in baccalau- 
reate math, and only 22 (5.1 percent) 
of that group passed the course. 
However, the percentage of students 



who took ELM more than once and 
who eventually passed ELM is 
roughly equivalent across the 
Marginal and Passing groups. Once 
they managed to pass the exam, 
roughly 40 percent of these students 
went on to succeed in the course, 
regardless of their initiai score. 

Key Findings on Student 
Performance 

• The initial score on the ELM exam 
is associated with baccalaureate 
math enrolling in and passing. 
Only 11.6 percent of ILE students 
pass math, compared to nearly 50 
percent of those who pass ELM 
on the first attempt and 29.5 
percent of Marginal students who 
nearly pass ELM on the first try. 

• Passing ELM greatly increases 
students chances of enrolling in 
and passing baccalaureate math. 
After four years, about 40 percent 
of those students who eventually 
pass ELM manage to pass bacca- 
laureate math — regardless of 
their initial score. 
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Key Findings on Student Retention 
Summer Bridge 

• Retention of Special Admit 
students compared favorably to 
that of the CSU system. Five-year 
retention/graduation rates were 
33.0 percent for Summer Bridge 
and 33.3 percent for the system. 

• Seven campuses retained or 
graduated over 40 percent of 
under-represented minorities: 
Fresno, Northridge, Bakersfield, 
Sacramento, Dominguez Hills, 
Humboldt, and Los Angeles. 

Intensive Learning Experience 

• Retention of ILE Special Admits 
compared favorably to CSU 
system figures. Five-year reten- 
tion/graduation rates were 31.8 
percent for ILE and 33.2 percent 
for the system. 

• Retention/ graduation rates of ILE 
students with ELM scores were 
higher than those of the ILE- 
Eligible comparison group (42.5 
percent to 40.5 percent). However, 
retention rates for all ILE students 
were less. 



• Relationships between ILE 
students' ELM and EPT scores 
were similar to those for Summer 
Bridge. ELM scores were posi- 
tively correlated to retention; EPT 
scores were not. However, EPT 
scores did have a linear relation- 
ship with rates of graduation. 
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